Diversity : Using the Engineering Design Process to Develop and Implement a Strategic Plan of Action for Undergraduate Diversity at the Institution Level

The under-representation of women and students of color in the undergraduate engineering population is a persistent and complex issue. The numerous “leaks” in the talent pipeline, along with the multifarious causes of under-representation , lead many institutions, including our own, to take a scattershot approach to recruiting and retaining diverse students in the undergraduate engineering population that may include extra-curricular K12 programming, college admissions scholarships, “gold shirt” programs, and wrap-around mentoring and academic support . While many of these programs have been shown effective in recruiting and/or retaining under-represented students into engineering, they are often implemented with little consideration to the scale or efficiency needed to achieve institution-level goals for undergraduate diversity, which assumes that such goals have even been clearly articulated in the first place. In this workshop, we propose and demonstrate the use of the Engineering Design Process (EDP) as an effective framework for goal-setting and developing targeted interventions to substantively advance undergraduate diversity at the institutional level. We adopted a 4-phase EDP that involves: (1) Defining the problem; (2) Generating multiple unique and viable concepts and selecting a final concept; (3) Detailed design and implementation of a final design; and (4) Design validation and iteration. This case study specifically details the use of Phase 1 through Phase 3 of the EDP for developing and implementing a strategic plan of action for undergraduate diversity at the institution level; and, to our knowledge, it represents the first attempt to use EDP in this context. Although this effort is still ongoing, we have thus far found EDP to be both efficient and effective in developing a clear plan of action related to undergraduate diversity. Our small working group, consisting of 8 faculty and staff members, initiated EDP in September 2016, concluding problem definition (Phase 1), concept generation and selection (Phase 2), and drafting of a final plan of action (Phase 3) within 6 months. This process included substantive buy-in from faculty uninvolved with the project as well as upper administration. One reason for this efficiency may be our own familiarity as engineers with EDP as well as the comfort of our peers and administrators with this process. We also developed several novel tools that may be useful, either stand-alone or as part of an institution’s diversity EDP. First, in defining diversity issues at our institution (Phase 1), we utilized publically available national databases to establish specific target values for student recruitment and retention within each engineering program at our institution. We found that the clarity of these targets resonated with faculty and administration, as well as the “friendly competition” fostered by intra and inter-departmental performance comparisons. A second valuable tool developed during this case study was the Diversity Intervention Graph (DIG), which allowed for easy visualization and, ultimately, selection of the vast array of potential interventions that could be applied towards solving diversity issues. In conclusion, we assert through this early-stage case study that EDP can be a roadmap for addressing issues of undergraduate diversity at the institution level. Given how daunting diversity issues can sometimes appear, we have found that framing and addressing this issue like engineers and explicitly using the EDP has made the process of goal setting, intervention, and evaluation remarkably clear. The overall process and specific tools presented in this case study may be easily extended to other institutions. Introduction The under-representation of women and racial minorities in the undergraduate engineering population is a persistent and complex issue. Taking a wide lens, this lack of diversity can be attributed to a variety of causes, including but not limited to cultural bias, lack of exposure or access, few role models, and general lack of interest in the discipline due to yet another range of factors like decontextualized instruction in core STEM courses and a perceived lack of societal impact relative to other disciplines . The numerous “leaks” in the pipeline, along with the shear variety of established causes, lead many institutions, including our own, to take a scattershot approach to diversity in the undergraduate engineering population. Through a patchwork of federal, state, and internal support, post-secondary engineering programs simultaneously offer intra and extra-curricular K12 programming, college admissions scholarships, “gold shirt” programs, and wrap-around mentoring and academic support . While many of these programs have proven to be effective in recruiting and/or retaining underrepresented students into engineering, they are often implemented with little consideration to the scale or efficiency needed to achieve institution-level goals for undergraduate diversity, which assumes that such goals have even been clearly articulated in the first place. In this paper, we propose and demonstrate that the Engineering Design Process (EDP) provides an effective framework for goal-setting and developing targeted interventions to substantively advance undergraduate diversity at the institutional level. We adopted a 4-phase EDP (Figure 1) that involves: (1) Defining the problem; (2) Generating multiple unique and viable concepts and selecting a final concept; (3) Detailed design and implementation of a final design; and (4) Design validation and iteration. This case study specifically details the use of Phase 1 through Phase 3 of the EDP for developing and implementing a strategic plan of action for undergraduate diversity at the institution level; and, to our knowledge, it represents the first attempt to use EDP in this context. Figure 1: A 4-Phase Engineering Design Process (EDP). Engineering Design Process Applied to Diversity The setting for this case study is a mid-sized, research-focused, land grant university on the US East Coast. Responding to institution-level priorities, the administration of the College of Engineering (COE) at this institution formed a working group, consisting of eight faculty and student-focused administrative staff with one faculty director, to focused on issues of diversity the COE undergraduate student body. The working group was provided a modest budget in its pilot year and direct access to institutional data, specifically from enrollment, admissions, and the registrar’s office. The working group adopted the Engineering Design Process (EDP) as a core philosophy for developing, implementing, and evaluating its strategic plan of action related to undergraduate diversity in COE. The outcomes of this process are presented in subsequent sections, using standard terminology related to EDP, which is underscored for emphasis in this case study. Phase 1: Problem Definition Project Scope The project scope is to achieve academic excellence by broadening participation within the COE undergraduate population. Given the present state of diversity in the College, the scope will presently encompass exclusively under-representation of women and under-represented racial groups (URGs, non-white and non-Asian) as a first effort towards diversification, recognizing that there are many other diverse groups, e.g., LGBTQ+, religious minorities, that will benefit from these early diversification efforts and will subsequently receive explicit consideration. Metrics & Target Values Metrics for gender and racial diversification were developed through benchmarking against other US engineering programs as well as researching underlying sociological phenomenon that result in persistent under-representation (Table 1). One of these phenomena is “critical mass,” which can be defined as sufficient representation of a minority population to self-perpetuate that population and affect cultural change within the broader community . Targets for critical mass are famously hard to pinpoint; however, it is generally accepted that 30% represents a valid “critical mass” for women in business, academia, and the sciences. For racial minorities in STEM, under-representation is so severe that 30% critical mass is unreachable without substantive shifts in secondary education practices; and a “skewed” distribution of 15% was targeted, which still represents substantial progress from present conditions. National benchmarks for the gender and racial composition and retention rates for undergraduate engineering students were also considered in developing metrics and associated target values for this plan (see Table 1). Using a published database containing demographic information for the graduating classes in every ABET-accredited engineering program in the US, target values were set for “average” to be the median and “excellent” to be the top quartile of programs nationally for gender and racial diversity in their graduating class. These target values developed for each engineering disciplines within the college separately, i.e., institution civil engineering vs. all civil engineering programs in the US. Similarly, a national report on student retention was data mined to establish “average” and “excellent” measures of student retention. Based on 6-year graduation rates, “average” retention was determined to be approximately 60% for majority (white male) and women students and 40% for URGs. “Excellent” retention rates were 70% with no disparities by race or gender. Table 1: Metrics table for design of strategic plan of action for undergraduate diversity. Metrics, target values, and current performance by engineering program of study are presented. Metric Target Value Program Current Performance Diversity of Graduating Class Women: 30% or 75th percentile nationally URGs: 15% or 75th percentile nationally %Students | Percentile Women URG Biomedical Engineering 43.3% | 57th 5.7% | 48th Chemical Engineering 28.8% | 32nd 4.3% | 30th Civil Engineering 20.9% | 9th 4.9% | 44th Computer Science 7.0% | 18th 5.9% | 32nd Computer Engineering 9.8% | 45th 6.2% | 43rd Electrical Engineering 11.9% | 46th 6.0% | 37th Environmental Engineering 41.9% | 23rd 11.5% | 34th Mechanical Engineering 17.2% | 74th 6.0% | 38th Student Retention 70% 6-year graduation rates with no difference by gender or race 6-Year Graduation Rate Majority Women URG Biomedical Engineering 61.0% 66.0% 57.2% Chemical Engineering 55.2% 51.8% 38.5% Civil Engineering 70.5% 63.2% 48.4% Computer Science 59.3% 42.9% 37.9% Computer Engineering 41.9% 27.8% 29.7% Electrical Engineering 63.0% 60.0% 55.9% Environmental Engineering 79.0% 76.7% 42.8% Mechanical Engineering 65.3% 68.0% 48.3% Current Performance COE’s current performance (see Table 1) was assessed relative to stated metrics by assembling multi-year institutional data from various internal databases, e.g., admission and registrar. These data showed that gender distribution of graduating classes by discipline strongly aligns with national trends, with biomedical, chemical, and environmental engineering exceeding 30% women; and computer science, computer engineering, and electrical engineering at less than 15% women. Biomedical, Chemical, and Environmental Engineering have achieved critical mass for women (30%), and Mechanical Engineering ranks just outside of the top quartile nationally in %Women (74 percentile). In terms of racial diversity, all COE programs fall far short of the top quartile nationally, ranging from 30 (Chemical Engineering) to 48 (Biomedical) percentile nationally; and they are also far below our “sufficient mass” target of 15% URG. COE’s current student retention rates also fall short of the target value for excellence (Table 2). COE’s Civil and Environmental Engineering have achieved 70% or greater 6-year graduation rates for the majority population, with Biomedical, Computer Science, Electrical, and Mechanical Engineering all performing at the national average of 60%. Computer Engineering (42%) and Chemical Engineering (55%) are presently below average even for the majority population. Most departments show little to no disparity by gender in terms of 6-year graduation rate, with the exception being Computer Science and Computer Engineering, which demonstrated 15%-point deficit for women. Nearly all departments, with the exception of Biomedical and Electrical Engineering, demonstrated racial disparities in 6-year graduation rates, with Computer Science, Computer Engineering, and Chemical Engineering performing below the national average of 40% 6-year graduation for URGs. No programs are presently close to achieving the target value of 70% retention for URGs. Further consideration of these data demonstrated that only relatively modest changes to recruitment and retention practices are necessary to meet the stated metrics. For instance, COE’s electrical engineering program, currently in the 46 percentile nationally, could boost their ranking into the top quartile by enrolling 3 additional women annually and increasing retention for all students from 60% to 70%. For URGs, similarly modest shifts in enrollment and retention are needed in order to achieve sufficient mass (15% URG) or top-quartile national performance. Given equivalent 6-year graduation rates, each discipline need only recruit 2-8 additional URGs in order to achieve sufficient mass or top-quartile performance nationally. An analysis of admissions data suggests that the depth of talent exists in COE’s applicant pool to meet recruitment goals. Demographics for the applicant pool are nearly identical with enrolled student populations within each engineering discipline. The yield rate for admitted students averages only 25% across disciplines, leaving a clear opportunity for targeted recruitment of women and URG students who already meet the COE’s high admission standards. For example, Computer Engineering averages 18 women accepted to the program annually and yields only 21% (3-4 students). Boosting yield to 27% (5 women students) – through direct outreach, financial aid, or targeted marketing to the admitted population – would allow the program to meet its diversity goals. Phase 2: Concept Generation & Selection Benchmarking The Phase 1 Problem Definition indicated a [design] opportunity for interventions focused on recruitment and retention of women and URGs. With this opportunity in mind, the working group undertook a comprehensive benchmarking process that involved two components. First, the group conducted an inventory of prior (<20 yrs) interventions within COE related to undergraduate student recruitment, retention, and diversity. Second, a literature review was also conducted, encompassing national reports of best practices in student recruitment and retention as well as case studies of individual institutions that have made notable progress in gender and/or racial diversity within their own undergraduate populations. From this benchmarking process, the working group developed the general consensus that comprehensive intervention is needed to diversify the undergraduate engineering population at a given institution. While a motivated group of individuals can certainly advocate for change, the institutions that serve as diversity benchmarks for the group, for instance, Carnegie Mellon and University of Maryland , achieved broad-based support from faculty and administration. Furthermore, these institutions do not consider “diversity” to be an issue separate from the other inner workings of an undergraduate program, particularly, enrollment management, financial aid, student advising, and, most importantly, teaching practice. Specific interventions from these benchmarks and others that were highlighted by the group included: (1) improving the quality of instruction for first and second year engineering and math/science core courses ; (2) segmentation of some introductory courses into novice and experienced sections to prevent stereotype threat ; and, to a lesser extent, (3) academic “gold shirt” programs aimed at addressing incoming student “deficiencies” in math and science preparation with additional remedial training . Convergent & Divergent Thinking Data from benchmarking were used to motivate a multi-week concept generation session within the working group that involved multiple iterations of convergent and divergent thinking. In the divergent thinking stage, group members were first tasked with generating as many interventions as possible related to undergraduate diversity. Over a three-week period, they solicited input from faculty, staff, and students within their specific engineering discipline. The group director also solicited input from faculty across COE at department faculty meetings. Ideas were also pulled from the aforementioned benchmarks as well as discussions with representatives from Admissions and student advising services. All potential interventions were recorded on note cards and brought back to the group. 103 potential interventions were generated by the 8 team members, with approximately 70% of these being unique. The working group then engaged in convergent thinking using a unique tool developed by the group specific to issues of undergraduate diversity (Figure 2). The tool, which we refer to as a Diversity Intervention Graph (DIG), explicitly considers two factors that the group deemed important in further strategic plan development, namely: (1) whether the intervention affected student recruitment and/or retention, which were found to be common themes in Problem Definition and Benchmarking; and (2) the intrinsic “cost” of undertaking the intervention, which was deemed important in prioritizing interventions with limited financial and human resources. DIG is an x-y plot that classifies potential interventions by: (x-axis) whether they most affect recruitment or retention practices, or both; and (y-axis) the amount of “activation energy” required to launch the intervention. “Activation energy” refers to funding, personnel, substantive changes to curriculum or teaching practice, or a combination of these factors. The working group engaged in an interactive convergent thinking session where interventions on notecards were physically sorted onto a large-format version of the x-y tool. The location of each intervention on the coordinate plane was determined by consensus, and the process was iterative. These data were then digitized and quantified, using a 5-point integer scale for the x-axis and 4-point for the y-axis. The number of interventions that mapped to a particular location on the DIG coordinate plane was counted and represented graphically as a bubble plot on the x-y plane. The size of the marker at each location on the DIG thus visually represents that intensity of group consensus around a particular intervention. Figure 2: The Diversity Intervention Graph (DIG), a novel tool for convergent thinking around diversity interventions. Results of the working group’s concept generation process are shown, representing 103 concepts generated by the group. The radius of each marker represents the number of concepts at that particular location. Concept Selection Once created, the DIG was an essential tool for concept selection in the development of the working group’s final design of a strategic plan of action related to undergraduate diversity (Figure 3). With feedback from COE administration, the working group reached the following two points of consensus: (1) final design should encompass the entire DIG concept space, with Admissions Events, Marketing, Background Research Admissions Events, K12 Outreach K12 Outreach Background Research, Marketing K12 Outreach Scholarships, Supplemental Courses/Tutoring Transfer Program, Admissions Criteria Interdepartmental Collaboration Student Organizations, Networking Internships Bridge Program, Admissions Criteria Student Organizations, Peer Groups, Networking, Cultural Change, Internships, Co-Residency Co-Residency, Academic Advisement, Peer Groups, Student Organizations, Networking Cultural Change, Curricular Change, Collaborative Workspaces, Internships, Academic Advising Interdepartmental Collaboration, Background Research Academic Advising, Supplemental Courses/Tutoring Cultural Change, Academic Advising, Supplemental Courses/Tutoring Cultural Change, Academic Advising, Supplemental Courses/Tutoring Recruiting Retention High Activation Energy Low Activation Energy the caveat that each individual concept location may not be addressed; and (2) the final design should prioritize “early wins,” meaning action should be immediately taken on elements of the plan that lead to COE meeting some of its design metrics quickly and with more limited resource allocation. These points of consensus led the group to sub-divide the DIG into a final design with three specific aims: (1) Refine student recruitment; (2) Strengthen student support services; and (3) Implement cultural and curricular change. Figure 3: The results of the concept selection process using the Diversity Intervention Graph (DIG). The final design includes three aims that encompass the entire DIG concept space. Phase 3: Design Details In accordance with the EDP model, the final design of the COE’s strategic plan for undergraduate diversity is focused on addressing the project scope and achieving the stated metrics of excellence in diversity (see Phase 1: Problem Definition). The final design consists of three aims and associated action items. A timeline for design implementation and a budget are also presented in the subsequent sections. Aims & Action Items The aims in the final design are summarized below, with specific action items presented in more detail in Table 2. • Aim 1 is to refine and continuously evaluate recruitment practices to increase enrollment of women and URGs in COE. COE faculty and staff will develop and adopt specific best practices around student recruitment, including not only incoming freshmen but also transfer students within other majors at the institution. These best practices fall into three categories, namely, marketing, direct outreach, and transfer management. • Aim 2 is to strengthen existing student support services and extracurricular programming around student recruitment, retention, and achievement, particularly for women and URGs. The thought here is that cultural factors that lead to under-representation and Admissions Events, Marketing, Background Research Admissions Events, K12 Outreach K12 Outreach Background Research, Marketing K12 Outreach Scholarships, Supplemental Courses/Tutoring Transfer Program, Admissions Criteria Interdepartmental Collaboration Student Organizations, Networking Internships Bridge Program, Admissions Criteria Student Organizations, Peer Groups, Networking, Cultural Change, Internships, Co-Residency Co-Residency, Academic Advisement, Peer Groups, Student Organizations, Networking Cultural Change, Curricular Change, Collaborative Workspaces, Internships, Academic Advising Interdepartmental Collaboration, Background Research Academic Advising, Supplemental Courses/Tutoring Cultural Change, Academic Advising, Supplemental Courses/Tutoring Cultural Change, Academic Advising, Supplemental Courses/Tutoring Recruiting Retention High Activation Energy Low Activation Energy Aim 1 Aim 2 Aim 3 Aim 1: Refine and continuously evaluate recruitment practices to increase diversity Aim 2: Strengthen existing student support services and extracurricular programming Aim 3: Educate and empower the faculty to implement cultural and curricular changes disenfranchisement of women and URGs happen both within and outside the classroom; and, given the existing resources within COE, it may be more feasible to initiate cultural change in student learning environments outside of the classroom before addressing more systematic curricular and pedagogical approaches (see Aim 3). This aim focuses on three areas: (1) establishing and disseminating best practices for student advisement across COE disciplines; (2) empowering and incentivizing student organizations to advance COE diversity goals; and (3) evaluating and strengthening existing COE-wide diversity initiatives, including our minority student and K12 outreach programs. • Aim 3 is to educate and empower the faculty to implement cultural and curricular changes that have been proven effective with diverse learners within and across undergraduate programs in COE. There are five interventions in this aim: (1) routinely engage and educate all faculty in constructive dialogue about diversity issues in the classroom; (2) ensure that first and second year courses are taught by the most effective faculty instructors; (3) incentivize the faculty to design and study programmatic and classroom-based interventions that address cultural barriers to success; (4) embed “alternative routes” for students through first and second year courses; and (5) study and optimize current admissions practices for promoting COE diversity efforts. Table 2: Detailed design of strategic plan for diversity developed using Engineering Design Process. Aims and associated action items are presented.